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the Bayesian Theorem and the degrees of behavioral objectives 
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combined with the utility values for each unit formed expected 
opportunity loss valiL^es in utiles from the conditional worth matrix. 
Utility and expected opportunity loss valuers indicated those 
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Aa Evaluation of Individualized Instructional Units Using the Bayesian Theorem 
in Two-Year Technical Institutes Designated as Developing Institutions 

Roger Woodbury, Rufus Swain 
Wilson County Technical Institute 

and 

Dove Henry Pate 
Clemson University 

An important operation of educational planning is the establishment of measuires 
of curriculum effectiveness. The types of curriculum effectiveness measures used in 
instructional systems include subjective and objective data consisting of personal 
values, degrees of obj ectiveness attained, and costs. Among the various planning 
or systems models used to analyze and to predict events in educational and other 
people-changing systems are the Bayesian theorem and cost/effectiveness analysis- 

The current study uses the Bayesian theorem and cost/effectiveness analysis to 
measure the short and long range effectiveness and minimum costs of individualized 
instructional units constructed by educational development officers for use v/ith 
low-income students at technical institutes designated as developing institutions. 
Students experiencing the individualized instructional units in the technical 
institutes responded to opinionnaires, measuring th' ir judgments of value of each 
instructional unit and questionnaires, measuring the degree of behavioral objectives 
achieved per unit. A cost analysis procedure developed calculated the costs for each 
instructional unit. Student value judgments of each instructional unit formed the 
prior distribution of the Bayesian theorem and the degrees of behavioral objectives 
achieved per instructional unit made up the sample distribution, P(A/B£), of the 
model. The variables combined in the joint probability distribution generated 
posterior probabilities (utility) P(B/A-i_) for each instructional unit. Costs per 
unit combined with the utility values for each unit formed expected opportunity 
loss values (EOL) in utiles frona the conditional worth matrix. 

Individual instructional unitG in given curricula with the highest utility 
values, P(B/A.)> and the lowest expected opportunity loss values in utiles were 
the optional choice in terms of cost and effectiveness. Utility and expected 
opportunity loss values generated over a three*-year period indicated those in- 
structional units with maximal effectiveness as well as those requiring revision. 
The current model is an effective, inexpensive planning procedure which dj^onstrates 
accountability in a variety of educational programs and systems. 
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Introduction 



An important operation of educational planning is the establishment 
of measures of curriculum effectiveness. The types of curriculum effec- 
tiveness or evaluation measures include subjective and objective data con- 
sisting of personal values and program costs. These data bases enable ed- 
ucational development officers as well as other professionals opportunities 
to assess the degree of attainment of educational objectives. Among the 
various models used in educational planning to analyze and to predict various 
events in educational and other people-changing systems are the Bayesian 
Theorem used as a curriculum or program evaluation procedure and cost/ 
effectiveness analysis used for curriculum or , '^^:ram revision and intra- 
program comparisons. 

The purpose for using the Bayesian Theorem or often called The Bayesian 
Statistical Decision Theory in curriculum evaluation is to estimate both 
short and long range program effectiveness. The Bayesian Theorem synthesizes 
prior inf oirmation in the form of personal probabilities and current infor- 
mation consistiTig of sample data to form a posterior probability indicating 
the degree of certainty of an event. The theorem with its use of prior or 
personalistic probabilities provides a model for the revision of judgments in 
':he light of new. information. The purpose for using cost/effectiveness 
analysis measurement is to relate costs of particular instructional components 
to their effectiveness or va^ue. Cost/effectiveness analysis identifies pro- 
gram components possessing increased effectiveness and diminishing costs. The 
theoretical basis of cost/effectiveness in public education is the equality 
of selling price and program costs. The current model of cost/effectiveness 
utilizes this "Break-even" concept. 
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ll\c BaycsLan Theorem 

A lengthy discussion of conditional probability and the Bayesian 
Theorem is not possible in the current paper. For an in-depth orientation 
to these topics, the writers recommend Novick and Jackson's (1974) and 
Schmitt's (1969) books. A succinct description of the Bayesian Theorem 
is reported below. 

The conditional probability of event A, given the sample space B is 
as follows: 

P(A/B) = P(A and B)^y P(B) t 0 (1) 
P(B) 

\^\en events A and B are independent, P(A and B) = P(A) . P(B), if 

P(B) 0 . therefore, 

P(A/B) = P(A) . P(B) = P(A), P(B) f 0 . (2) 
P(B) 

Then, given the definition of conditional probability and the union 
of mutually exclusive events, the theorem can be formulated. Given the 
mutually exclusive events Bj, B2, - - - B^, each with non-zero prc-babilit;^ ^ 
wliosc union I3 the sample space* then 

P(A) = P(A/Bi) . P(Bi) + P(A/B2) . P(B2) ... (3) 

n 

. . . P(A/Bn) . P(Bn) = H P(A/Bi). P(Bi) 

i=l 

because the probability of event B , given A, is 

P(Bi/A) = P(A/Bi) . P(Bi) (4) 
P(A) 

n 

and since P(A) = H P(A/Bi) . P(Bj^) 
i=l 
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then Bayes' Theorem follows by substituting equation 3 in the 

denominator of the equation: 

P(Bi/A) « P(A/Bi) , ^>(Bi) (5) 
n 

£ P(A/Bi) . P(Bi) 
i=l 

Some basic definitions will clarify. the theorem. The probability of 
event, P(Bj[), is defined as an a~priori probability, which is a subjective 
or personal probability based on the rational judgment of the individual 
assigning the probability. In using the theorem, the individual assigns a 
subjective estimate, P(Bj[), about an event, then makes an observation, 
P(A/Bi), and calculates the a-posteriori probability, P(B/Ai). The obser-- 
vation, PCA/B^), is the probability of A under condition B^ . Collectively, 
these probabilities are the observed sample distribution. The posterior 
probabilities, P(B/A-j^), calculated from the Bayesain Theorem are used in 
subsequent experiments to update probabilities. That is, the posterior 
probabilities of experiment I become the ptior probabilities, P(Bi), of 
experiment II, etc. 

Applied to curriculum evaluation, the theorem generated utility and 
cos-t /effectiveness values on individualizeci instructional units developed 
by educational development officers in four North Carolina technical institutes. 
The evaluation study spanned three years with support from Title III, The Basic 
Institutional Development Program. 

More specifically, let each individualized instructional unit or pro- 
gram element be designated Bj, B2, B3 . . . B^^. Each curriculum is des- 
ignated Aj, A2, A3 . . . . A^. Then, the symbol, P(A/B^), is defined cs the 
probability of program A given B,. . Each value of P(A/Bi) is actually a scaled 
value (questionnaire) measuring the degree of behavioral objectives achieved 
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per instructional unit. These values represent the sample distribution, 
P(A/Bj^), of the Bayesiian Theorem. Then, each program element within a 
curriculum forms the prior distribution, P(Bi). Actually, these are 
judgments of program worth or value measured on an interval scale (opin- 
ionnaire), forming the prior distribution of the Bayesian analyses. The 
degree of specific behavioral objectives achieved per unit combined with 
the normalized value judgments through the Bayesian Theorem generates the 
pos'tcrfor probabilities ot utility values. The utility value of each unit 
is intergrg^ed v/ith unit costs in the expected opportunity loss model pro- 
viding cost/effectiveness indices. Basically, the expected opportunity loss 
(EOL) is the product of the. Bayesian utility values and the absolute value 
of the difference betwes-n costs and selling price, known as conditional 
opportunity loss (COL). Tne expected opportunity loss (EOL) value for 
a given program element or individualized instructional unit is the sum 
of the products of each probability and the cou .clonal opportunity loss 
(COL). If costs and selling prices are equal, then procedures for det- 
ermining eicpected opportunity loss (EOL) values are trivial. The basic 
model for combining the utility of each program element; let's say Pi, 
which costs dollars is 

n (Si - ci) 
1=1 

utiles, the minlnum expected loss for a set of elements or units in a pro- 
gram. The minimum expected opportunity loss is the best decision or 
optimal solution if one is selecting instructional units or program elements 
for continued use. If the differences bet^^een costs and selling prices 
are unequal, more analyses are needed 

7 
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Curriculum Evaluation 

The evaluation format utilizing the Bayesiarj Theorem and a cost/ 
effectiveness model is based on Tanner's (1971) program and the writers' 
applied research in curriculum, programs in four technical institutes of 
The North Carolina Community College system. The discussion, thus far, 
has not covered the use of these models in a program context. While the 
Bayesian and expected opportunity loss cost/effectiveness models themselves 
generate interest to researchers and enlightened administrators, their use 
as a viable curriculum and program evaluation procedure encourages sys- 
tematic planning and decision-making in all areas of instructional systems. 

Four technical institutes in North Carolina formed a consortium and 
received funding from Title III, The Basic Institutional Development 
Program, for curriculum development. The mission of the curriculum develop- 
ment program was to develop individualized instructional units in the curric- 
ulum programs of the four technical institutes: (1) Wilson County Technical 
Institute, (2) Pitt Technical Institute, (3) Edgecombe Technical Institute 
and (4) Roanoke-Chowan Technical Institute. Having the designation of 
Developing Institution by the Title 111 program, the four institutes used 
the curriculum development program to alleviate weaknesses in their in- 
structional systems. Educational development officers used the Systems 
Approach to Individualized Instruction in developing the instructional units. 
Basically, the systems approach to developing individualized instructional 
units involves six compcr.ents: (1) rationale, (2) objectives, (3) pre-test, 
(4) learning activities, (5) post-test and (6) revision. ' Theoretically, 
individualized instructional units based on the systems ^proach permit 
students to engage in learning activities leading to the attainment of 
specific performance objecti/es. In addition, the focus of the instruct- 
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ional units is on what the student has learned using rhe instructional units. 
One implication made from the use of this approach is accountability for 
results in curriculum programs of developing institutions. 

The philosophy of pi.ogram accountability inherent in the management- 
by-objectives and results procedure used by the consortium's coordinating 
institution, Wilson County Technical Institute, prompted the application 
of the above curriculum effectiveness models on individualized instructional 
units In curriculum programs. The rationale for using the models was to 
use an evaluation procedure determining program utility (value) and to in- 
tcrgrate the variables of utility and costs. The three sets of major 
variables used in the models arc the degree of objectives achieved, forming 
the sample distribution, P(A/Bi); the relative contribution or value of 
each instructional unit in a given instructional program, forming the prior 
distribution, P(Bi); and the cost (dollars) per unit. 

The primary assumption of the models as Tanner ^(1971) indicated is 
that utility (the posterior probability) is a function that reveals math- 
ematically the relationship between values or preferences and objectives 
under the condition of uncertainty. Utility (posterior probability) is 
removed from the classical concept involving only dollars values. The 
judgments of program worth or value are determined for each instructional 
unit on an interval scale as measured by an opinionnaire. The degree of 
specific objectives achieved per unit as measured by a questionnaire is 
combined with the normalized value judf^inents through the Bayesian process 
to determine utility or the posterior probability. The posterior pro- 
bability (utility) is then intergrated with the costs of each instructional 
unit in the expected opportunity loss model providing cost/effectiveness 
indices. Specifically, the posterior probabilities, PCB/A^) of each unit 
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are multiplied by the conditional opportunity loss (COL) values (in dollars) 
to forra expected opportunity lotis (EOL) values (utiles). The objective 
of the expected opportunity loss (EOL) analyses combining utilities and 
conditional opportunity loss values is to determine the minimum cost and 
mximum utility of instructional units when compared with others in the 
same n.-rlculum program. The unit with the lowest EOL value in utiles is 
the most valuable component in terms of cost and effectiveness. Since one 
central concern of administrators is the use of instructional components with 
minimal costs and maximum effectiveness, tAe current models are appropriate 
for educational planning. Such was the case in assessing the value and 
coGts of individualized instructional units developed in the four institutes. 

Over a three year period (1973-197A - 1975-1976) educational develop- 
ment officers developed over 100 individualized' instructional units for a 
large number of vocational and technological curricula in the four technical 
institutes. Procedures used to develop, evaluate, and revise the instruct- 
ional units followed a standardized format described below. 

Initially, the educational development officers and faculties cooperatively 
developed instructional units for particular curricula. The systems approach 
Wlfch it3 six compoTients was -th^ ftameworlc for instructional development". A 
cost analysis procedure used by the educational development officers det- 
ermined the cost (dollars^ of each instructional unit The cost procedure 
considered instructional and staUf salaries, sunplies, and materials, main- 
tainence costs and travel in the calcualtion of a total cost per unit figure. 

The educational development officers and faculties used the new in- 
structional units in regularly scheduled courses at the institutes. In some 
situations, educational developm.ent officers* actually taught the classes 
since they desired to be effective teachers as well as curriculum developers. 

10 
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III eacli courno of lUv Mutluntu wlu) u.u'd v-acli ot tl\o no.w Instruct iouaL uull: 
Sonu» iivit ruot lonaL ur.Lts h.ul littlo faro validity wltli MtudiMitfi b»/raniH? 
of varloun wtsikiUJMtU!.-; In .heir structun' ^e^c hlt;li lovol of difficulty, 
vai.;uc» content, and hroad objectives) • Thi»a« unltii wrv, wlthdrav^a from tlie 
cour.sea and not used In the evalu.it Inn utudy. 

In the i'prlui; of each of the thrife yeara, tlie educational developinenl 
oftlcer;; and Title III coordinator randooly selected 77 individualized unll 
for analysLi?. Costti f each unit matched coMta or^lnally tabulated on tlu. 
cost analysis shurits. Then, for each Individualized instructional unit 
selected, the Title III coordinator developed two instruiuents to assess tJ\^ 
value or contribution of the Instructional unit in t'.. context of the part 
icular curriculum proc::.%n. First, an opinionnaire inoasurin^^ rhe dejjree of 
contribution (value) each unit has to the successful achicvemt?nt of a part 
icular course consists of a 32 point scale similar to the one introduced 
by Likert (1967). The data frora the 32 point scale are .ornialized per n 
instructional units in each curriculum program and represents the prior 
distribution* P(B|) (Appendix A). Second, a questionnaire measuring the 
degree of beha/iorai objectivas achieved for each unit r^pt^sents the per- 
centage achieved per program element (instructional unit) (Appendix A). 
Each percentage beconies a component in the sample distribution, P(A/Bi). 
It is important to note that only one objective per unit is necessary tc 
evaluate the instructional units. Also, each unit may have more than one 
objective. 

Next, the educational development officers randomly selected students 
from curricula who used the individualized instructional units chosen. 
In the spring of each of. the three years (1973-1974 - 1975-1976), the 

11 
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fitUii*-nt?i f4?^hn!ttnl and rilUu! out, Uuth thy uplaLymmlri^a m\d qucat tonnalrou • 
A Moftwarti pro^.tMm on tho lUryordan and cont /^•Ctrxt i vim\omji modobi Injitalhul 
In tiio computor faoUltli»:i at North Carolina State Unlvurnlty by 
1)1 Horbort Kirk calculated tlic data reported below. 

Oatn aualyaivi of all tht! lant ructional units used over tlio three year 
period (1973-197^ - 1975-F*/6) in beyond the icopc of the current paper. 
To Illustrate tlie curriculum ef fectlvenous procedures described above, llayesian 
aaJ cost /effect Ivene.is data of cl^ht indivLdualUed instructional units from 
three curriculum projjrams depict effectiveness measurament in curriculum 
pro^V"<^fns. These data are reported iti Tablco 1 and 2. 
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Table I: Utilities of Individualized Instructional Units over three Program Evaluations (1973-1974 - 1975-1976) 





^ Instructional 


Initial 
Prior 

Distribution 
P(Bi) 


Posterior 
Distribution 
(Utility) 
1973-1974 (a) 


Posterior 
Distribution 
(Utility) 
•197H975 (b) 
P(B/Ai) 


Posterior 
Distribution 
(Utility) 
1975-1976 (c) 
P(B/Ai) 


Mean 

Utility 

P(B/Ai) 


LligilSn 


LOMUniCaLiUub 

Skills I 


40 


44 


.4- 


.56 


.48 




tOmlUUUl C d L iUUb 

Skills II 


.35 


.35 


.35 


.43 


.38 




wlllmUU 1 Ua L i U U 0 

Skills III 


25 


.23 


.22 




".23 


Radiology 


Kadiograpnic 
Anatomy 




fin 


.64 


.67 


.63 




Radiographic 
Physics 


.45 


.39 




.32 


.36 


Trig- 
onometry 


Introductory 
Terms. 


.33 


.34 


.35 


.34 


.34 




Trig. Tables 


.33 


.33 


.33 


.33 


.33 




Trigonometric 
Functions and 
the Slide Rule 


.33 


.33 


.3i 


.33 


.33 



(a) Posterior distribution in 1973-197^, study and the prior distribution in the 1974-1975 analysis. 



(b) Posterior distribution in the 1974-1975 study and the prior distribution in the 1975-1976 analysis. 

(c) Posterior distribution in the 1975-1976 study and to be used as tke, prior distrubition in 1976-1977 analysis, 
- Instructional unit not used in 1975-1976 analysis. . U 



Table 2: Costs and Expected Opportunity Loss Values for Individualized Instructional Units over three 
Program Evaluations (19'3-1974 - 1975-1976) 



Course ■ Instructional jCost EOL EOL EOL Mean 

Unit Value (a) Value (b) Value (c) EOL 

1973-1974 1974-1975 1975-1976 



English CoMunications $900.00 196.78 200.92 .239.40 212.37 
Skills I 

Comunications 

Skills II $350.00 376.10 371.44 310.60 352.71 

.. ■:. Comunications 

Skills III ■ $950.00 223.90. 228.56 - 228.23 



Radiology Radiographic 
Anatomy 



$800.00 278.22 252.87 230.52 260.53 



Radiographic 

Physics $100.00 421.78 447.13 469.48 446.13 



Trig- Introductory 
ononietry Terras 

]_5 Trigonometry 
Tables 



$150.00 19.81 19.61 19.65 19.69 



$150.00 19.81 19.61 19.65 19.69 



Trigonometric 

Functions and , 

the Slide Rule $ 90.00 40.19 40.39 40.00 40.16 
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Inspection of the utilities and cost/effectiveness vali-as (in utiles) 
in the instructional units of the three courses reveal some rather interesting re- 
sults. In the English course, for example, the first instructional unit with 
the highest mean utility value and , the lowest mean expected opportunity loss 
value is clearly the most valuable in terms of cost and ef iectiveness . The 
other two units in the English course were not as valuable in terms of cost 
and effectiveness, since .both had lower utilities and higher expected oppor- 
tunity loss values. Actually, the third unit. Communications Skills III, 
was revised by the educational development officers and will be used again 

by the institutes. 

In the radiology course, the first unit, radiographic anatomy, had the 
higher utility values and the lower expected opportunity loss values when 
compared with its companion unit, radiographic physics. Interestingly ' 
enough, the radiographic anatomy package utilizing expensive multi-media 
sound and slide presentations was not commercially produced, but the product _ 
of educational development officers. The commerically developed, less ex- 
pensive radiographic physics unit failed to have greater cost/effectiveness 
with a mean utile value of A46.13. 

Pata collecfced on the -trigonometry course tended to he. very similar 
with no clear-cut patterns emerging. While the first instructional unit, 
introductory terms, had the highest utility value when compared with the 
other two units, the mean expected opportunity loss values of the first two 
units were equal and considerably lower than the value for the third unit. 

Another interesting condition is the similarities of the utilities and 
expected opportunity loss values over the three year span. One explanation 
for the similarities is the consistent up-dating of the prior distribution 
P(Bi), with the posterior distribution, P(B/Ai) from the previous year's 

17 
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analysis and the use of the identical cost figure of each instructional 
unit. The frequent up-dating of the prior distribution, P(B-i^) , through 
posterior probabilities, P(B/Ai) , merely add data to the group's value 
judgiEents and make the prior distribution, P(B/Ai), less depp.ndent on 
tiie original prior distribution, PCB^) . 

The evaluation procedure delivers a systematic plan based on both sub- 
jective probabilities and costs to administrators and curriculum specialists. 
The procedure produces data on how students think they will function as 
a result of experiencing specific instructional units. Cognitive data such 
as standardized achievement and criterion-referenced tests may be used 
in the sample distribution, PCA/B^) , of the Bayesian model. Also, the 
Bayesian and cost/effectiveness models may be applied to other evaluation 
programs in postsecondary institutions. 

One result of the current study is that a systematic attempt to measure 
curriculum effectiveness in programs funded by Title III, the Developing 
Institutions Act, was made. The authors contend that the assessment* of 
program effectiveness is a giant step toward demonstrating program 
accountability with federal monies. In -the vernacular, trjing to develop 
curricula with maximum effectiveness with a minimum of costs is "Trying 
to bring off what you stated in the project you would bring off". 
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APPENDIX A 
Roanoke-Chowan Technical Institute 



Questionnaire 

Course: Physics 

Directions: Below is a list of instructional units in Physics which you have 
studied. For each unit there is an important objective listed which you have 
worked toward. Circle (%) or check df) the percentage of the behavioral objective 
you believed you achieved. 



Unit: Sound Waves 

Objecti>/e: The student will be able to define the nature of sound, Use its 
Chaifacteristics and solve scientific problems relating to sound. 

(Percent Achievad) 



0% 



10% 



20% 



30% 



A0% 



0.% 



70% 



80% 



90% 



100% 



Unit: Harmonic Motion & Waves 

Objective: The student will be able to demonstrate an understanding for properties 
of simple harmonic motion and waves, list types and characteristics of 
waves, and solve mathematical problems pertaining to waves. 

(Percent Achieved) 



0% 



10% 



20% 



30% 



40% 



50% 



60% 



70% 



80% 



90% 



100% 
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Roanoke-Chowan Technical Institute 
Opinionnaire 



Course: Physics 



Directic-ns: Below is a list of instructional units on Ph3'sics you have studied. 
For each unit, rank each on the scple according to the degree which the unit has 
contributed to your successful achievement of the Physics Course. 



Unil:: Sound Waves 
ver;^ little 

0 1234567 



(Circle Nvmber) • 

a great deal 
8 9 10 11 12 13 14 15 16 



UuJt: Karmonic Motion Waves 
V? '^7 ..ittle 

0 1 2 3 4 5 6 

I 



(Circle Number) 



10 11 12 13 14 



a great deal 
15 16 . 
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